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Remarks 

Claims 1-13 are pending in the application. Claims 1-13 are rejected. All 
rejections are respectfully traversed. 

The invention rank orders multimedia content. Image or video multimedia 
content is segmented to extract objects. Features of the objects are then 
extracted and associated to produce content entities. The content entities are 
coded to produce directed acyclic graphs of the content entities, each 
directed acyclic graph representing a particular interpretation of the 
multimedia content. Attributes of each content entity are measured and the 
measured attributes are assigned to each corresponding content entity in the 
directed acyclic graphs to rank order the multimedia content. 

Claims 1-13 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Yeo et al. ("Rapid Scene Analysis on Compressed Video") in view of 
Ganser et al. (U.S. Patent 4,953,106). 

Yeo is a well-known paper in the art of scene change detection in 
compressed video. Yeo takes as input a compressed video and segments the 
video into shots. Yeo defines a shot at page 533, left col., second paragraph, 
below: 

For video, a common first step is to segment the videos into 
temporal "shots " each representing an event or continuous 
sequence of actions. A shot is what is captured by a camera 
between a record and a stop operations. Further scene analysis 

and further down, \ 
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would be displayed. The boundaries between video shots are 
commonly known as scene changes and the act of segmenting 
a video into shots is called scene change detection. There 

Yeo is entirely directed to segmenting a video into shots for scene change 
detection, see page 3, right col., first full paragraph, below: 

In this paper, we propose several fast algorithms that operate 
directly on Motion JPEG 1 [I] or MPEG [2] compressed 
videos for temporal segmentation and flashlight detection. 

A person of ordinary skill in the art would never confuse a shot with an 

object. Even Yeo distinguishes an object from a shot, see page 540, left col., 

section D, below: 

video sequences. Unless the shot consists of little or no camera 
motion and no object motion, one frame may not be sufficient 
for such purposes. 

Therefore, the Examiner's assertion that Yeo teaches segmenting the 
multimedia content to extract objects can never be true, because Yeo 
explicitly teaches segmenting a video into shots, and defines a shot as "what 
is captured by a camera between a record and a stop operations." The 
Examiner has confused scene change detection as in Yeo with object 
extraction as in claim 1. 

Features of the objects are extracted and associated to produce content 
entities. As stated above, Yeo never teaches extracting objects and therefore, 
does not describe extracting and associating features from objects to produce 
content entities. The Examiner cites page 533, left column: 
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interactive multimedia systems. There is an urgent need to 
automatically extract key informations from images and videos 
for the purpose of indexing, fast and easy retrievals, and scene 
analysis. 

That only teaches that extracting key information is important for indexing, 
retrieving, and analysis. . The Examiner is respectfully requested to 
specifically point out which words mean extracting and associating features 
of the objects to produce content entities for the purpose of ordering, as 
claimed. 

Further, the Examiner points to section VII of Yeo as describing measuring 
image attributes for image compression. Claimed is measuring content 
entity attributes, the content entities derived from extracted and associated 
features of the objects extracted from the multimedia content. Yeo does not 
teach content entities. 

Gasner describes a method for drawing directed graphs providing reduced 
crossings and improved picture quality to produce a drawing of the graph. 
Gasner has nothing to do with producing a directed acyclic graph. Gasner 
takes as input a directed graph, in the form of a node interconnection listing, 
which can be acyclic, and generates the graph drawing. Node positions in 
the drawing are assigned in a way that reduces crossings of the lines 
connected the nodes. 

Claimed is coding the content entities to produce directed acyclic graphs of 
the content entities, each directed acyclic graph representing a particular 
interpretation of the multimedia content. The measured attributes are 
assigned to each corresponding content entity in the directed acyclic graphs 
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to rank order the multimedia content. Gasner initially assigns arbitrary ranks 
to nodes of a graph as follows: 

Col. 5 

edge is non-negative. To compute the rank function 
from the final T (block 208), for each tree in T, do a 50 
standard breadth first search through the tree, starting 
at some arbitrary node and assigning it an arbitrary 
rank. Each time a new node is scanned, it is assigned the 
rank of the node currently being visited* plus or minus 
1, depending on whether the scanned node is the head 55 
or tail node, respectively, of the edge between it and the 
visited node. Finally, the rank assignment for each tree 
is shifted to make the smallest rank equal to 0. 

Gasner generates a drawing of the graph that has few edge crossings and can 

be easily understood by a user viewing the graph drawing, see col. 6, below: 

106 of FIG, 6, Consequently, optimal rank assignments 
are important not only for their contribution to drawing 
quality, but also for minimizing the number of dummy 
nodes and reducing the number of edge crossings and 
m the cost of subsequent passes. This substep also reserves 



Gasner assigns arbitrary ranks to nodes of a graph and shifts the ranks of the 
nodes according to a rank function to produce a drawing of the graph that 
has few or no line crossings. Claimed is assigning attributes to content 
entities to rank order content entities. Gasner is useless for making the 
invention obvious. 



In claim 2, the measured attributes include intensity attributes. The 
Examiner cites Yeo at section VII, page 543. Section VII describes DCT- 
based image compression. As stated above, Yeo never describes measuring 
attributes of content entities derived from extracted and associated features 
of the objects extracted from the multimedia content, as claimed. The same 
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is true for claims 3-5, where the measured attributes include direction 
attributes, spatial attributes, and temporal attributes, respectively. 

In claim 6, the measured attributes are arranged in an increasing rank order 
and in claim 7, the measured attributes are arranged in a decreasing rank 
order. The Examiner's statement that Gasner "would have provided the 
capability for ensuring the initial node ordering of a directed tree has no 
crossings," is non-sequitor. The Examiner's use of Gasner as a reference is 
totally off the mark. Ensuring the initial node ordering of a directed tree 
has no crossings is entirely irrelevant to what is claimed. 

Claim 8 recites traversing the multimedia content according to the directed 
acyclic graph and the measured attributes assigned to the content entities. 
Gasner describes assigning 2D spatial coordinates for a position for nodes 
based on an optimized rank assignment that reduces crossing of the lines 
connecting the nodes. That has nothing to do with traversing multimedia 
content, as claimed. 

Claim 9 recites summarizing the multimedia content according to the 
directed acyclic graph and the measured attributes assigned to the content 
entities. Determining optimal rank assignments of spatial locations of nodes 
for drawing a graph has absolutely nothing to do with multimedia content 
summarization. There is no relationship between the two whatsoever. The 
Applicants are so thoroughly perplexed by the Examiner's assertion, it is 
requested the Examiner provide a more understandable explanation of what 
he understands summarizing multimedia content means. 
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In claim 10, the multimedia content is a three dimensional video sequence. 
As stated above, Yeo never extracts objects from a video sequence as 
claimed. Yeo segments a compressed video into shots. 

In claim 11, nodes of the directed acyclic graphs represent the content 
entities and edges represent breaks in the segmentation, and the measured 
attributes are associated with the corresponding edges. Gasner only teaches 
nodes of directed acyclic graphs. Columns 9 and 10 of Gasner describe 
drawing a graph. The Examiner's assertion that Gasner describes nodes 
represent content entities and edges represent breaks in segmentation is pure 
conjecture because there is absolutely no description of what a node 
represents in Gasner and Yeo never teaches content entities as claimed. The 
Examiner is requested to specifically point out exactly where Gasner 
describes nodes represent anything at all, and particularly content entities as 
claimed. 

In claim 12, at least one secondary content entity is associated with a 
particular content entity, and wherein the secondary content entity is selected 
during the traversing. The Examiner refers to a section of Gasner that 
describes assigning spatial coordinates to nodes based on rank and priority. 
There is no description of associating content entities, or nodes for that 
matter. According to Gasner, a priority of a node is defined as a weighted 
sum of its out-edges or in-edges. The Examiner is requested to explain what 
that has to do with what is claimed. 

In claim 13, a summary of the multimedia is a selected permutation of the 
content entities according to the associated ranks. Gasner never summarizes 
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anything. Gasner draws graphs. Gasner assigns node positions on the graph 
so that lines connecting the nodes do not cross. The Applicants have no idea 
how the Examiner came to the conclusion that Gasner is relevant to the 
invention. 

It is believed that this application is now in condition for allowance. A 
notice to this effect is respectfully requested. Should further questions arise 
concerning this application, the Examiner is invited to call Applicant's 
attorney at the number listed below. Please charge any shortage in fees due 
in connection with the filing of this paper to Deposit Account 50-0749 . 

Respectfully submitted, 

Mitsubishi Electric Research Laboratories, Inc. 




Andrew J. Curtin 
Attorney for the Assignee 
Reg. No. 48,485 



201 Broadway, 8 in Floor 
Cambridge, MA 02 139 
Telephone: (617) 621-7573 
Customer No. 022199 
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